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Abstract: Nowadays, a wide range of new possibilities for the construction of intelligent systems 
for the control of technological processes are developing through the use of artificial neural 
networks. The article describes the widespread use of symbols through artificial neural networks 
in issues such as recognition, prediction and diagnosis, optimization, signal processing under the 
influence of noise, and discusses the main parameters of the artificial neural network by the 
authors. 
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Although science and information and communication technologies have developed rapidly and 
in the era of globalization leading to the fourth industrial revolution, technologies have 
penetrated deeply into all spheres of human activity, man and machine have not yet fully 
interacted on an equal footing. 


One of the most pressing issues in the developed countries of the world is the widespread use of 
modern information technologies and artificial intelligence in public administration, economy, 
industry, social protection, education, medical diagnostics, agriculture, defense and security, 
tourism and many other areas. 


Today, a wide range of new capabilities for building intelligent process control systems are being 
developed through the use of artificial neural networks and are widely used in issues such as 
image recognition, prediction and diagnostics, optimization, signal processing under the 
influence of noise. As a result, the development of machine learning in storing and processing 
large amounts of data is of great importance. Artificial neural networks consist of a number of 
inputs and outputs, consisting of a set of interconnected neurons that perform nonlinear 
transformations. Deep learning of artificial intelligence systems using multi-layered artificial 
neural networks is the key to the success of artificial intelligence. That is why in this article we 
found it necessary to cover the issues that are necessary in the use of neural networks. 


It is known that the basis of all artificial neural networks - neurons. 


Nowadays, logical thinking, empirical skills and imagination are important for modern software 
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and hardware engineers in building efficient artificial neural networks for information 
processing. We would like to briefly discuss the components and operation of artificial neural 
networks. 


1. An extended model of an artificial neural network: 
> weight coefficient; 

aggregate alarm function; 

activation function; 

output function; 


error function and inverse distribution value; 


VV VV WV 


teaching function. 

2. Artificial neural network architecture. 
> properly distributed neural networks; 
> feedback recurrent neural networks. 
3. Artificial neural network training: 
> teacher training; 

> teaching without a teacher; 

> assessment of teaching. 

4. Rules of teaching: 

> Hebb's rule; 

Hopfield's rule; 

“Delta” rule; 

gradient rule; 


stochastic rule; 


VV VV Y 


competition training. 


The artificial neural network at the same time works with many neurons. Each neuron has its 
own synaptic weight. The weight coefficient characterizes (w) the interdependence of neurons 


in the layers and it determines the magnitude of the communication efficiency of the elements. 
We know that each element of the neural network operates over a discrete time and forms the 
resulting signal based on the signal received. The weight coefficients, which are the main 
character for the output signal, are amplified, while the weights of insignificant effects are 
compulsorily reduced. This determines the intensity of the input signals. Weight coefficients can 
vary according to teaching methods, neural network architecture, and training rules. The sum of 
the input signals generated is converted to the output signal in the activation function. Using the 
activation function to detect the output signals of the artificial neural network, the total signal is 
compared to a certain limit. Several types of activation functions are currently used in practice. 
Outcomes are calculated using activation functions in the form of the most sigmoidal and 
hyperbolic tangents (Table 1). The activation functions in this view are nonlinear, and the given 
multidimensional function can be ground with arbitrary precision per unit cross section. As a 
result, the output layer also has a nonlinear characteristic (Figure 1). 
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Table 1. Activation functions 


Name Formula Value range 
1 
Si idal S)= 0,1 
igmoida f(S) T (0, 1) 

ee 

Hyperbolic tangent | f(S)=—,——, | Cl, 1) 
e +e 

a) b) 
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Figure-1. Appearance of activation functions: a) sigmoidal; 


b) hyperbolic tangent 


The use of the above activation functions gives clear results and it is widely used. But different 
activation functions can be selected for different artificial neural networks. 


Output function. Compared to a biological neuron, each artificial neuron has a single output 
signal, which is interconnected with hundreds of other neurons. Basically, the output signal is 
directly proportional to the result of the activation function. In some network architectures, the 
results of the activation function are modified to increase the interaction between adjacent 
neurons. Efforts are made to reduce the effect of neurons of the artificial neural network with 
weak signals. This can occur between neurons located in the same layer or in different layers. As 
a result, it helps to determine which neurons are active and the output signal as well as 
participating in the learning process. 


Error function and inverse distribution value. In all controlled artificial neural networks, the 
difference between the output signal value and the required value in “training”. The error 
function is selected according to the architecture of the artificial neural network. Basically, the 
deviation error value is used directly, sometimes quadratic or cubic values of error are also used 
in some paradigms. The error is applicable to all layers and is affected depending on the type of 
activation function. The results will be applied to the next training period. 


Teaching function. The purpose of training an artificial neural network is to correct the values of 
the weight coefficients in all neurons and layers based on a specific algorithm to obtain the 
required output signal. The training process is divided into 2 types: controlled and uncontrolled. 
In controlled learning, the neuron requires a set of training data or an observer that ensures the 
effectiveness of the network results. In uncontrolled learning, the system organizes learning 
based on a self-determined algorithm. 


Artificial neural network architecture. Neurons in artificial neural networks form information 
processing systems that allow the model to adapt effectively to constant changes in external 
influences. As a result of a process in an artificial neural network, input signals are converted to 
output. This process is determined by the neural network architecture, the characteristics of the 
neural elements, and the control and synchronization of information flows between neurons. An 
important factor in the effectiveness of an artificial neural network is the organization of the 
optimal number of neurons and layers and the types of connections between them. In describing 
neural networks, a number of terms are used that may have different interpretations in different 
sources, in particular: 
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1) neural network structure - a method of connecting neurons in a neural network; 
2) neural network architecture - neural network structure and types of neurons; 
3) Neural network paradigm - a method of teaching and use. 


The neural network paradigm sometimes includes the concept of architecture. The different 
paradigms of the neural network can be realized on the basis of a single architecture, and vice 
versa. 


We know that neural network architecture can theoretically be divided into two groups. If each 
neuron of a neural network is connected only to neighboring neurons, these are neurons that are 
not fully connected (Figure 2). When the inputs are connected to the outputs of other neurons, 
the fully connected neurons are networks (Figure 3). 


Figure-2. Incompletely connected neural Figure-3. Fully connected neural networks 
networks 


In doing so, the neurons merge into layers. The external input vector connects the input layer of 
the neural network and the output signal to the output of the last layer. In addition to the input 
and output layers, there are also one or more latent layers in the neural network that are not 
exposed to external influences. The connection between neurons in different layers is called 
projectile. Connections from the input layers to the output layers are afferent, the reverse 
direction of the connections is efferent. The interconnection of neurons in a single layer is called 
lateral. The architecture of a multilayer neural network can be divided into two groups. 


1) correctly distributed networks (connected in series to one side) - perceptrons, Back 
Propagation, oppositely distributed network, Kohonen network; 


2) Inversely distributed networks (recurrent connection) - Hopfield network, network of 
adaptive resonance theory, two-way network. 


They are typical architectural networks that have many modifications and can be components of 
other networks. Properly distributed networks are classified as static because the given inputs of 
the neurons receive an input signal vector independent of the previous state of the network. 
Inversely distributed networks are dynamic because the input of neurons changes instantly due to 
feedback, which leads to a change in the state of the network. The uniqueness of the artificial 
neural network, which is considered an analogue of biological neural networks, is the ability to 
learn from the examples that make up the training. 


Artificial neural network training. Training can be conducted with or without a teacher. 


Teacher training. In neural networks, controlled training is used to solve many problems, and the 
output signal is constantly compared with the required value. The weight coefficients are initially 
selected randomly, but during subsequent iterations they are updated to match the required and 
current output. During neural network training, the current errors of all neurons are minimized as 
a result of continuous updating of the weight coefficients to achieve network accuracy. In 
controlled training, the neural network must be trained before use. The training period can be 
long, especially in systems where the processing power is incompatible, lasting several hours. 
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Training is considered complete when the desired output signal and sufficient accuracy are 
generated in a given sequence of input signals. Upon completion of training, the weight 
coefficients of the neurons are accepted for reuse. A key component to the successful operation 
of a neural network is the representation and coding of input and output data. Neural networks 
only work with digital data, so input data needs to be changed. Computers can easily transfer 
data from sensors to machine format. 


If, after training in a controlled learning process, the neural network effectively processes the 
data of the training set, it is also important to work with data not used in training. If 
unsatisfactory results are obtained during the training trial, the process will continue. The test 
should provide not only the study of the data of a particular training set, but also the general laws 
that may be contained in the data. 


Evaluating the effectiveness of neural network training depends on several controlled factors. 
According to the theory of neural network teaching, we can cite three main components of 
teaching: data volume, learning patterns, and the complexity of computations. Data volume 
refers to how many patterns a network can remember and what decision boundaries it can create. 
The complexity of the training patterns determines the number of training sets required for the 
network to achieve generalizability. The complexity of the calculation depends on the power of 
the computer processor. 


The article provides detailed information about the capabilities of artificial neural networks, 
types of architectures, inputs and outputs, as well as the main parameters, terms and their 
essence. The information provided is useful for those conducting research in this area. 
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